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Abstract — In the field of image analysis and processing, 
the post section of having a data record plays a vital role. 
The research work carried out for analyzing a crack 
image in an oil pipeline titled “Image Analysis and 
processing using mathematical morphological operators 
and high frequency filter for pipeline crack 
measurement” had a difficult phase of saving the data for 
future study. To overcome this issue a technique to 
preserve the image data is handled with the concept of 
big data analysis. A pipeline scanned for quality 
maintenance sends numerous pictures, where the pixel 
data is converted to binary data and then these are 
calculated using the mathematical morphological 
operator based on erosion and corrosion of the images. 
These data are saved for further reference where the 
normal method of data saving using any hardware device 
was a big threat for loss of data and the cot to maintain 
the system is too high. To overcome this issue image data 
is been tried to save through the big data technique. 
Keywords — Bigdata, Cluster Analysis, Machine 
Learning, Prescriptive analytics, Supervised Learning. 

I. INTRODUCTION 

“Modern Machine Learning is devoted to deriving value 
from data, not jamming the airlocks. ” 

Arthur C Clark’s the above quote is to be proved soon, 
using the Big data enabled Machine Learning. Image 
processing can be analyzed using Machine Learning 
based methods like clustering and classification. 
Clustering methods helps to learn the relationship of 
image intensities for segmentation without the concept of 
domain knowledge. Image specifying features are taken 
to analyze the pipeline crack. Large volume of data is 
required for tough cracking images against imaging relics, 
intensity and also for shape variations. On the other hand 
Classification methods help to learn from the sample or 
training data given. This method is performed by the data 
set generated from the digitized image of a crack. 



Fig.l: Pipeline image with a crack (pic courtesy 
Engineering Mechanics Corporation of Columbus) 


In the implementation of Machine learning Approach 
algorithm either supervised or unsupervised algorithm can 
be applied. But as far as this research work is concerned 
supervised learning with cluster analysis technique has 
been implemented. So it is best to follow the supervised 
machine learning technique to extract the data from the 
cluster images of cracked oil pipeline. The new 
development in modem computers has helped to exhibit 
systems with good performance and storage capacity. 
Machine learning system is used to learn data from the 
past to the current and helps in a comparative study of the 
data available. This is carried out by the effectiveness of a 
machine learning model depending on the high speed, 
volume and high variety of data and big data helps 
perfectly to carry out the task. 

In this research paper the task of converting the existing 
binary data storage to big data through machine learning 
is to be achieved. 
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Fig. 2: (pic courtesy Engineering Mechanics Corporation 
of Columbus) 
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n. BIG DATA 

Big data means a massive volume of data that consists of 
both structured and unstructured data that is too large to 
process using normal database and software techniques. 
Image processing of a pipeline deals with volume of data 
than the usual capacity of holding both the past and 
current values of the captured image. 

Big data analytics extracts meaningful patterns from 
massive input data from the image analyzed for decision 
making and predictions. 

m. MACHINE LEARNING 

Machine learning is the modern science of finding 
patterns and making predictions from data based on work 
in any kind of image processing methods where the 
volume of data is too big. 

Big data is implemented with Machine learning methods 
on image data where deep and predictive insights need to 
be uncovered from data sets that are large, diverse and 
fast changing. Machine learning easily identifies the 
dataset when compared to the existing traditional 
methods. 

When Machine learning algorithm and big data are 
combined they do predictive analytics, deep learning, 
prescriptive analytics, and machine learning. Image 
analysis and processing for a pipeline with a large set of 
data helps to make a clear and effective data set to 
identify the cracks in the image at ease. 

The ability to automatically apply complex mathematical 
calculations to big data in any type of data repeatedly and 
quickly is the recent application in the field of big data 
and machine learning. This definition helps to reconstruct 
the data model in the research paper of “Image Analysis 
and processing using mathematical morphological 
operators and high frequency filter for pipeline crack 
measurement”, where the mathematical morphological 
operated data from the cluster image is processed using 
the big data and machine learning concept. This is a big 
revolution in the image analysis and processing field 
where the data for future use will be preserved for any 
further requirement. 

IY. METHODOLOGY 

To design an efficient machine learning system, the data 
from the image should be recognized efficiently. 
Algorithms to be implemented should be the best fit. 
Automation and iterative processing should be effective. 
Scalability should be measured accordingly to know the 
progress of the implementation and overall it should be an 
ensemble model. 


V. CONCLUSION 

The Machine learning model designed with big data helps 
to maintain a good track of the data that is analyzed using 
the image from the pipeline. This paper will be 
implemented with 5,000 sample sets of data taken from 
image and then later depending upon its successful 
performance the sample set will be doubled or increased 
more to get a faster and effective result. 

In the overall performance of this method of sample 
treating has paved way for more applications through big 
data and machine learning . 
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